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Abs tract 

A final  report  on  research  activities  of  ONR  contract  "Comnuni- 
ication  Methods",  covering  the  period  of  1972  to  1976,  is  presented.  Topics 
of  research  include  digital  and  analog  coninunication  in  the  presence  of  inter - 
ferences;coding  of  information  sources;  and  models  of  stochastic  processes 
for  signals  and  noises.  A list  of  research  publications  and  a list  of  per- 
sonnel supported  in  this  period  are  given.  A sumnary  of  research  activities 
and  their  relevance  are  also  included. 
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I.  INTRODUCTION 

In  this  technical  report  entitled,  "Final  Report  on  Communication 
Methods",  research  activities  covering  the  period  from  1972  to  1976  are 
described.  These  investigations  were  conducted  under  the  support  of  the 
Office  of  Naval  Research,  Contracts  N00014-69-A-0200-4043  and  N00014- 
75-C-0528,  and  Task  NR-042-285.  Dr.  B.J.  ^lcDonald,  Director  of  the 
Statistics  and  Probability  Program,  was  the  scientific  officer  in  charge 
of  these  contracts. 

In  Section  II,  a List  of  Research  Publications  that  appeared  in  the 
period  1972-1977  is  given.  In  Section  III,  a List  of  Personnel  Supported 
in  the  period  1972-1976  is  presented.  In  Section  IV,  a Summary  of  Research 
Activities  and  their  relevance  are  discussed. 
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II.  LIST  OF  RESEARCH  PUBLICATIONS 

The  following  research  publications,  which  appeared  in  the  period 

1972-1977,  are  listed  in  chronological  order. 

1.  K.  Yao,  "A  Compact  Form  of  Error  Expression  for  Pulse  Comnunication  System 
with  Intersymbol  Interference,"  Conference  Record  of  the  International 
Conference  on  Communications,  pp.  46.11-46.14,  June  1972. 

2.  W.D.  Hopkins,  Structural  Properties  of  Rate  Distortion  Functions,  Ph.D. 
thesis,  UCLA,  June  1972.  Technical  Monograph,  Dept,  of  System  Science, 
UCLA,  June  1972. 

3.  K.  Yao,  "On  Minimum  Average  Probability  of  Errors  Expression  for  a Binary 
Pulse-Conmunication  System  with  Intersymbol  Interference,"  IEEE  Transac- 
tions on  Information  Theory,  Vol.  IT-18,  pp.  528-531,  July  1972. 

4.  J.H.  Slaton,  Optimum  Sonar  Performance  in  the  Presence  of  Reverberation 
and  Noise,  Ph.D.  thesis,  UCLA,  Sept.  1972.  Technical  Monograph,  Dept, 
of  System  Science,  UCLA,  Sept.  1972. 

5.  J.H.  Slaton  and  K.  Yao,  "Optimum  Sonar  Signal  Design  for  Matched  Filter 
Detectors,"  Technical  Monograph,  Dept,  of  System  Science,  UCLA,  Sept. 

1972. 

6.  J.H.  Slaton  and  K.  Yao,  "The  .Amplitude  and  Spectrum  of  Surface  Reverber- 
ation as  Determined  from  Surface  Resonance  Theory,"  Technical  Monograph, 
Dept,  of  System  Science,  UCLA,  Sept.  1972. 

7.  J.W.  Collier,  Numerical  Study  of  Minimum  Probability  of  Error  Expression 
in  a Digital  Comnunication  System  with  Intersymbol  Interference,  M.S. 
thesis,  UCLA,  Dec.  1972.  UCLA  Technical  Report,  Eng. -7327,  Jan.  1973. 

8.  H.  Tan  and  K.  Yao,  "Information  Rates  of  Countably  Infinite  Alphabet 
Memoryless  Sources,"  Proceedings  of  the  Seventh  Annual  Princeton  Confer- 
ence on  Information  Sciences  and  Systems,  March  1973,  p.  249  (Abstract). 

9.  K.  Yao  and  L.B.  Milstein,  "Maximum  Likelihood  Receiver  for  Binary  Signals 
with  Intersymbol  Interference  in  Gaussian  Noise,"  Conference  Record  of 


the  International  Conference  on  Communications,  June  1973,  pp.  141-144. 


-4- 


10.  H.  Tan,  On  the  Evaluation  of  Rate  Distortion  Functions,  Ph.D.  thesis, 

UCLA,  June  1973.  UCLA  Technical  Report,  Eng. -7356,  July  1973. 

11.  H.  Tan  and  K.  Yao,  "Information  Rate  of  a Memoryless  Gaussian  Source 
under  an  Absolute  Magnitude  Criterion,"  Abstracts  of  Papers  of  the 
1973  International  Symposium  on  Information  Theory,  June  1973,  p.  B3-3. 

12.  K.  Yao,  "A  Representation  Theorem  and  Its  Applications  to  Spherically- 
Invariant  Random  Processes IEEE  Transactions  on  Information  Theory, 

Vol.  IT-19,  September  1973,  pp.  600-608. 

13.  K.  Yao  and  H.  Tan,  "Some  Comments  on  the  Generalized  t-ower  Bound  for 
Stationary  Finite-Alphabet  Sources  with  Memory,"  IEEE  Transactions  on 
Information  Theory,  Vol.  IT-19,  November  1973,  pp.  815-817. 

14.  K.  Yao,  "Series  Expansions  for  Random  Processes  on  a LCAG,"  Proceedings 
of  the  Eighth  Annual  Princeton  Conference  on  Information  Sciences  and 
Systems , March  1974,  p.  58. 

15.  E.  Shomash  and  K.  Yao,  "On  the  Structure  and  Performance  of  a Linear 
Decision  Feedback  Equalizer  Based  on  the  Minimum  Error  Probability  Cri- 
terion," Conference  Record  of  the  International  Conference  on  Conwunica- 
tions,  June  1974,  pp.  25F.1-5. 

16.  J.  Kawamura,  Adaptive  Detection  in  the  Presence  of  Unknown  Noise,  Ph.D. 
thesis,  UCLA  June  1974.  Technical  Monograph,  Dept,  of  System  Science, 

UCLA,  June  1974. 

17.  K.  Yao,  "Properties  of  Rate -Distort ion  Functions,  Part  2:  Rate-Distortion 
Functions  with  Absolute  Value  Distortion  Measure,"  in  Source  Coding /F.rgodic 
Theory  Research  Seminar  Lecture  Notes,  Ed.  by  R.M.  Gray,  Stanford  Univer- 
sity Tech. Report  No.  6503-2,  September  1974,  pp.  93-101. 

18.  K.  Yao,  "Series  Expansions  and  Applications  of  Second  Order  Processes  on 
a Locally  Compact  Abelian  Group,"  Abstracts  of  Papers  of  the  1974  Inter- 
national Symposium  on  Information  Theory,  October  1974,  pp.  24-25. 

19.  H.  Tan  and  K.  Yao,  "Rate-Distortion  Functions  of  Countably-Infinite  Alpha- 
bet Memoryless  Sources,"  Abstracts  of  Papers  of  the  1974  International 
Symposiun  on  Information  Theory,  October  1974,  pp.  63-64. 
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20.  I.  Rubin  and  K.  Yao,  "A  Distortion -Measure  Sensitivity  Analysis  of 
the  Rate-Distortion  Function,"  Proceedings  of  the  Eighth  Hawaii  Inter- 
national Conference  on  System  Sciences,  January  1975,  pp.  83-85. 

21.  H.  Tan  and  V.  Yao,  "Evaluation  of  Rate -Distort ion  Functions  for  a Class 
of  I.I.D.  Sources  under  an  Absolute  Magnitude  Criterion,"  IEEE  Trans- 
actions on  Information  Theory,  Vo.  IT-21,  January  1975,  pp.  59-64. 

22.  R.M.  Tobin  and  K.  Yao,  "Error  Bounds  for  Digital  Comnunication  Systems 
in  the  Presence  of  Gaussian  Noise  and  Cochannel  Interference,"  Proceed- 
ings of  the  1975  Conference  on  Information  Sciences  and  Systems,  April 
1975,  p.  196  (Abstract). 

23.  H.  Tan  and  K.  Yao,  "Rate -Distort ion  Functions  of  Countably- Infinite  Alpha- 
bet Memoryless  Sources,"  Information  and  Control,  Vo.  27,  March  1975, 

pp.  272-288. 

24.  K.  Yao  and  R.M.  Tobin,  "A  New  Class  of  Upper  and  Lower  Error  Bounds  for 
Digital  Comnunication  Systems  with  Intersymbol  Interference,"  Conference 
Record  of  the  1975  International  Communication  Conference,  June  1975, 
pp.  35.11-35.15  . 

25.  K.  Yao,  "Encoding  of  a Continuous  Source  by  Twisted  Modulation,"  Pro- 
ceedings of  the  1975  Allerton  Conference  on  Circuit  and  System  Theory, 
October  1975,  pp.  481-492. 

26.  K.  Yao  and  L.B.  Milstein,  "On  ML  Bit  Detection  of  Binary  Signals  with 
Intersymbol  Interference  in  Gaussian  Noise,"  IEEE  Transactions  on  Conmu- 
ications,  Vol.  COM-23,  September  1975,  pp.  971-976. 

27.  K.  Yao  and  R.M.  Tobin,  "Moment  Space  Upper  and  Lower  Error  Bounds  for 
Digital  Systems  with  Intersymbol  Interference,"  IEEE  Transactions  on 
Information  Theory,  Vol.  IT-22,  January  1976,  pp.  65-75. 

28.  T.Y.  Yan,  Moment  Space  Error  Bounds  for  Digital  Comnunication  Systems 
with  Intersymbol  Interference  Based  on  Nth  Moment,  M.S.  thesis,  UCLA 
December  1975.  UCLA  Technical  Report  Eng. -7608,  January  1976. 

29.  K.  Yao,  "Error  Probability  of  Spread  Spectrum  Multiple  Access  Communi- 
cation Systems,"  Proceedings  of  the  1976  Conference  on  Information 
Sciences  and  Systems,  March  1976,  pp.  67-72. 
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30.  K.  Yao  and  H.  Tan,  "On  Rate-Distortion  Functions  of  I.I.D.  Sources 
under  an  Absolute-Magnitude  Criterion,"  Proceedings  of  the  1976  Confer- 
ence on  Information  Sciences  and  Systems,  March  1976,  p.  61  (Abstract). 

31.  K.  Yao  and  R.M.  Tobin,  "Multi-Dimensional  Moment  Space  Error  Bounds  for 
Digital  Systems  with  Multiple  Interferences,"  Abstracts  of  Papers  of  the 
International  Symposium  on  Information  Theory,  June  1976,  p.  69  (Abstract). 

32.  K.  Yao,  "Performance  Bounds  on  Spread  Spectrum  Multiple  Access  Communi- 
cation Systems,"  Proceedings  of  the  International  Telemetry  Conference, 
September  1976,  pp.  317-325. 

33.  K.  Yao,  "Quadratic -Exponential  Moment  Error  Bounds  for  Digital  Communica- 
tion Systems,"  Conference  Record  of  the  Tenth  Annual  Asilomar  Conference 
on  Circuits,  Systems,  and  Computers,  November  1976,  pp.  99-103. 

34.  R.M.  Tobin  and  K.  Yao,  "Upper  and  Lower  Error  Bounds  for  Coherent  Phase- 
Shift  -Keyed  Systems  with  Cochannel  Interference,"  IEEE  Transactions  on 
Communications , Vol.  CCM-25,  February  1977,  pp.  281-287. 

35.  K.  Yao,  "Error  Probability  of  Asynchronous  Spread  Spectrum  Multiple 
Access  Communication  Systems ,"  IEEE  Transactions  on  Communications , Vol . 
CCM-25,  Special  Issue  on  Spread  Spectrum  Communications , Summer  1977 
(full-length  paper). 

36.  K.  Yao,  "Moment  Space  Error  Bounds  in  Digital  Communication  Systems," 
Proceedings  of  the  NATO  Advanced  Study  Institute,  Aug.  1977. 

37.  Data  Communication  - Intersymbol  Interference,  Ed.  by  K.  Yao  (Dowden, 
Hutchinson  and  Ross,  publ.)  Sumner,  1977. 

38.  K.  Yao  and  H.  Tan,  "Evaluation  of  Rate -Distortion  Functions  for  I.I.D. 
Sources  with  Support  on  Subsets  of  1R  under  an  Absolute -Magnitude  Cri- 
terion," submitted  to  the  IEEE  Transactions  on  Information  Theory. 
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III.  LIST  OF  PERSONNEL  SUPPORTED  BY  THE  CONTRACT 

The  following  people  performed  work  and  were  partially  supported  by 
the  contracts  during  the  period  1972  to  1976.  The  names  are  listed  in 
alphabetical  order. 

1.  J.W.  Collier,  M.S.  degree,  received  1972. 

2.  W.D.  Hopkins,  Ph.D.  degree,  received  1972. 

3.  M.  King,  Ph.D.  degree  candidate,  1976. 

4.  J.  Kawamura,  Ph.D.  degree,  received  1974. 

5.  S.  Reisenfeld,  Ph.D.  degree  candidate,  1975-1976. 

6.  E.  Shomash,  Ph.D.  degree, received  1973. 

7.  J.H.  Slaton,  Ph.D.  degree,  received  1972. 

8.  H.  Tan,  Ph.D.  degree,  received  1973. 

9.  R.M.  Tobin,  Ph.D.  degree,  received  1976. 

10.  T.Y.  Yan,  Ph.D.  degree  candidate,  1976. 

11.  K.  Yao,  Associate  Professor  and  Principal  Investigator,  1972-1976. 
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IV.  SM1AKY  OF  RESEARCH  ACTIVITIES 

In  this  section,  we  present  a brief  summary  of  the  research  activities 
that  have  been  preformed  in  the  period  from  1972  to  1977.  The  theoretical 
and  practical  relevances  of  these  research  results  are  discussed.  Our  dis- 
cussions are  grouped  along  the  line  of  three  fields  of  interest.  The  first 
field  is  in  digital  and  analog  communication  in  the  presence  of  interfer- 
ences. The  second  field  is  in  coding  of  information  sources.  The  third 
field  is  in  models  of  stochastic  processes  for  signals  and  noises. 

Now,  consider  our  first  field  of  research  interest.  In  an  idealized 
environment,  the  most  basic  aspect  of  statistical  communication  theory  is  to 
decide  optimally  the  presence  or  absence  of  a signal  in  random  noise. 

In  practice,  due  to  the  presence  of  limitations  in  real  transmission  chan- 
nels, interferences  induced  by  the  signals  are  generally  present.  Thus, 
these  interferences,  which  are  random,  combine  with  the  ever  present  addi- 
tive random  channel  noise  to  degrade  the  performance  of  the  overall  system. 
There  is  a significant  difference  in  performance  for  such  a system  that  has 
been  optimized  as  compared  to  one  that  neglects  the  presence  of  such  inter- 
ference. 

In  our  sonar  and  reverberation  channel  study,  from  [4]  ("Optimum  Sonar 
Performance  in  the  Presence  of  Reverberation  and  Noise"),  relevant  past  tech- 
niques are  reviewed  critically  and  some  new  practical  results  are  obtained. 
The  modem  techniques  of  detection  and  signal  design  are  useful  to  practical 
sonar  problems  only  if  the  statistical  characteristics  of  undersea  reverber- 
ation channels  are  known.  In  the  past,  these  characteristics  were  derived 
from  parameters  on  an  ad-hoc  point -scatterer  model.  In  [6]  ("The  Amplitude 
and  Spectrum  of  Surface  Reverberation  as  Determined  from  Surface  Resonance 
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Theory*'),  we  removed  the  need  for  these  expedients  with  respect  to  the  surface 
component  of  reverberation  by  developing  a direct  relation  between  the  rever- 
beration characteristics  and  the  physical,  acoustical,  and  geometrical  para- 
meters. Our  theoretical  results  agree  well  with  the  limited  available  exper- 
imental data.  In  [5] ("Optimum  Signal  Design  Methods  for  Sonar  Detection"),  we 
presented  an  optimum  signal  design  procedure  for  a sonar  design  which  oper- 
ates in  a background  of  reverberation  and  noise,  and  which  used  a practical 
matched  filter  detector.  Results  from  our  procedure  were  compared  with  the 
conceptually  superior  (but  much  more  complicated)  reverberation  filter.  We 
have  shown  in  practical  situations  that  the  reverberation  filter  may  be  super- 
ior only  in  a limited  range  of  conditions.  Thus,  our  results  are  of  consid- 
erable interest  from  the  practical  sonar  design  point  of  view. 

In  any  digital  conmunication  transmission  channel,  there  is  only 
available  a finite  bandwidth  of  transmission  frequency.  Thus,  any  finite 
time-duration  pulse  representing  some  digital  data  upon  passage  through  the 
transmission  channel  will  have  infinite  time  duration.  Consequently,  result- 
ing pulses  representing  adjacent  data  will  cause  intersymbol  interferences. 
Consequently,  intersymbol  interference  is  an  extremely  active  topic  of  re- 
search and  development  in  communication  technology' . This  is  due  to  the 
increasing  transmission  traffic  of  digital  data  by  conmercial  and  governmental 
agencies.  In  [1]  and  [3]  ("A  Compact  Form  of  Error  Expression  for  Pulse  Com- 
munication System  with  Intersymbol  Interference"  and  "On  Minimum  Average 
Probability  of  Error  Expression  for  a Binary  Pulse-Conrninication  System  with 
Inters/mbol  Interference"),  we  have  obtained  a new  analytical  equation  for 
the  optimum  coefficients  of  a linear  equalizer  under  a minimum  probability 
of  error  criterion.  Before  the  existence  of  our  result,  the  exact  closed- 
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form  expression  for  the  minimum  probability  of  error  has  been  considered 
to  be  "nearly  impossible  to  compute”  (Lucky,  Satz,  and  Weldon,  "Principles 
of  Data  Communications,"  McGraw-Hill,  1968,  p.104). 

In  [ 1 5 J ("On  the  Structure  and  Performance  of  a Linear  Decision  Feed- 
back Equalizer  Based  on  the  Minimum  Error  Probability  Criterion"),  we  used 
the  above  technique  for  a linear  decision  feedback  equalizer.  A decision 
feedback  equalizer's  performance  can  be  considerably  better  than  that  of 
a linear  equalizer  with  only  slight  increase  in  complexity.  Past  work  by 
others  on  linear  and  decision  feedback  equalizers  have  been  based  mostly  on 
mean-square  or  peak  distortion  criteria.  For  digital  comnuncation  system 
analysis,  the  probability  of  error  is  of  course  the  true  criterion  of  impor- 
tance. In  practice,  equalizers  designed  on  the  basis  of  minimum  probability 
or  error  criterion  are  only  slightly  superior  to  those  based  on  other  cri- 
teria. Nevertheless,  we  feel  our  work  on  these  equalizers  provides  a firm 
foundation  for  the  analysis  of  these  systems.  Yet,  despite  the  frequent  use 
of  linear  and  dec is ion -feedback  equalizers  in  practice,  it  is  known  that  the 
structure  of  the  optimum  equalizer  should  be  nonlinear  and  based  on  the 
maximum -likelihood  criterion.  In  [9]  and  [26]  ("Maximum -Likelihood  Receiver 
for  Binary  Signals  with  Intersymbol  Interference  in  Gaussian  Noise"  and 
"On  ML  Bit  Detection  of  Binary  Signals  with  Intersymbol  Interference  in 
Gaussian  Noise"),  we  considered  such  an  equalizer.  Besides  various  general 
basic  properties,  for  the  first  time  we  have  shown  in  these  papers  that  it 
is  possible  to  obtain  the  performance  (i.e.,  probability  of  error)  of  this 
highly  complicated  nonlinear  receiver  by  analytical  approximation  without 
tedious  numerical  simulations. 

A basic  problem  in  digital  communication  theory  is  the  evaluation  of 
error  probability.  In  most  problems  of  realistic  interest,  it  is  generally 
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not  possible  to  evaluate  it  analytically.  Of  course,  one  can  perform  a 
simulation  of  the  system.  Often  a simulation  is  expensive  and  generally 
one  obtains  no  real  insight  into  the  problem;  consequently,  one  is  forced 
to  use  various  bounding  techniques.  While  standard  results  of  Chebychev 
inequality  and  Chemoff  bound  may  be  adequate  for  information  theory  pur- 
poses, where  one  is  generally  only  interested  in  asymptotically  large 
samples,  these  techniques  often  do  not  yield  adequately  tight  bounds  for 
practical  digital  communication  systems.  We  have  used  results  from  the 
theory  of  moment  spaces  and  have  obtained  new  upper  and  lower  bounds  on 
error  probability  that  are  significantly  tighter  than  previously  known 
bounds  for  several  practical  digital  communication  system  problems.  The 
basic  moment  space  isomorphism  theorem  states  that,  for  a random  variable 
of  distribution  of  compact  support,  the  compact  body  M,  defined  as  the 
moments  of  the  r.v.  with  respect  to  n arbitrary  continuous  functions, 
is  isomorphic  to  the  convex  hull  of  the  curve  traced  out  by  the  function 
values  of  these  n functions.  While  this  basic  theorem  is  by  Karlin  and 
Shapley,  we  believe  that  it  has  not  been  used  in  information  theory  nor  in 
communication  system  analysis.  In  [27]  ("Moment  Space  Theory  Upper  and 
Lower  Error  Bounds  for  Digital  Cormunication  Systems  with  Intersymbol  Inter- 
ference") and  [24]  ("A  New  Class  of  Upper  and  Lower  Bounds  for  Digital  Com- 
munication Systems  with  Intersymbol  Interference"),  we  considered  a linear 
equalizer,  based  on  any  error  criterion,  and  used  the  above  moment  space 
isomorphism  theorem.  We  showed  various  previously  known  bounds  on  error 
probability  can  be  reduced  to  some  of  our  special  cases.  Furthermore,  we 
constructed  a class  of  "exponential  kernel"  bounds  which  can  be  orders  of 
magnitude  tighter  than  previously  known  bounds.  Thus  , these  bounds  are  not 
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only  tight  but  are  reasonably  simple  to  use.  In  [34]  ("Error  Bounds  for 
Digital  Comnunication  Systems  in  the  Presence  of  Gaussian  Noise  and  Co- 

channle  Interference"),  we  used  the  above  technique  for  a class  of  cochannel 

/ 

satellite  and  microwave  comnunication  link  analyses.  These  problems  are  of 
considerable  practical  importance  and  our  results  are  again  significantly 
tighter  than  previously  known  results. 

In  recent  years,  there  has  been  considerable  interest  in  communication 
systems  with  spread  spectrum  and  multiple  access  capabilities.  The  spread 
spectrum  effect  is  clearly  of  use  for  secure  comnunication  purposes.  For 
satellite  applications,  spread  spectrum  modulation  yields  a conceptually 
simple  solution  to  the  requirement  of  lowering  radio  frequency  interferences 
of  one  user  by  another.  On  the  other  hand,  multiple  access  capability  is  of 
benefit  when  many  users  share  the  bandwidth  and  transmission  capability  of  a 
comnunication  system,  or  for  a satellite  comnunication  system  with  a single 
wide  band  repeater.  In  such  situations,  a code  modulation  asynchronous  spread 
spectrum  multiple  access  (SSMA)  system  is  applicable  in  a network  of  low-cost 
mobile  ground-bas^i  and  air-bounced  users  requiring  no  network  control.  The 
SSMA  technique  has  also  been  used  in  multi-user  computer  communication  net- 
works. In  any  case,  for  all  these  and  other  related  SSMA  systems,  the  exact 
evaluation  of  error  probability  has  been  considered  a formidable  task.  Error 
probability  obtained  by  a complete  simulation  of  such  systems  may  involve 
considerable  computational  cost  and  need  not  yield  much  insight. 

In  [29]  ("Error  Probability  of  Spread  Spectron  Multiple  Access  Communi- 
cation Systems"),  [32]  ("Performance  Bounds  on  Spread  Spectrun  Multiple  Access 
Comnunication  Systems")  and  [35]  ("Error  Probability  of  Asynchronous  Spread 
Spectrum  Multiple  Access  Comnunication  Systems"),  we  obtained  upper  and  lower 
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bounds  of  these  spread  spectrum  multiple  access  communication  systems  by 
using  the  space  technique.  Derivations  for  the  second  moment,  fourth 
moment,  single  exponential  moment,  and  multiple  exponential  moment  are 
given  in  terms  of  the  partial  cross  correlations  of  the  codes  used  in  the 
system.  Error  bounds  based  on  the  use  of  these  moments  are  obtained.  By 
using  a sufficient  number  of  terms  in  the  multiple  exponential  moment, 
upper  and  lower  bounds  can  be  made  arbitrarily  tight.  In  that  case,  the 
error  probability  equals  the  multiple  exponential  moment  of  the  multiple 
access  interference  random  variable. 

It  is  clear  that  many  other  closely  related  digital  communication 
systems  and  digital  signal  processing  error  probabilities  can  be  bounded 
using  this  technique.  We  believe  this  moment  space  bounding  technique 
has  significant  impact  on  the  theoretical  understanding  of  error  probabil- 
ity in  digital  signal  analysis.  At  the  practical  computational  level, 
our  moment  space  bounds  are  reasonably  attractive  and  in  all  cases  are 
equal  to  or  tighter  than  any  other  known  error  bounds. 

In  the  second  field  of  research  interest,  we  considered  coding  of 
information  sources.  Due  to  the  world -wide  "information  explosion",  there 
is  a real  need  for  data  reduction  prior  to  information  storage,  processing, 
and  transmission.  We  have  been  mainly  interested  in  the  theoretical  as- 
pects of  this  problem.  A rate-distortion  function,  originated  by  Shannon, 
is  the  minimum  information  rate  needed  to  specify  a source  between  the  input 
and  output,  subject  to  some  tolerable  distortion.  Thus,  this  quantity  is 
basic  to  the  study  of  source  encoding.  In  [2]  ("Structural  Properties  of  'late- 
Distortion  Function"),  we  have  shown  that  the  basic  concept  of  "innovation” 
(i.e.,  "new  information"),  oriy  ed  by  Wold  and  comnonly  used  in  linear 
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and  nonlinear  filtering  theory,  is  also  useful  in  the  evaluation  of  the  rate- 
distortion  function.  In  general,  it  is  difficult  to  calculate  the  rate-dis- 
tortion function  of  a source  and  the  standard  approach  is  to  calculate  the 
more  easily  obtainable  Shannon  lower  bound.  Often  this  lower  bound  is  quite 
close  to  the  actual  function  and  sometimes  it  is  actually  equal  to  it.  Up 
to  now,  for  discrete  sources,  the  Shannon  lower  bound  approach  has  been  applic- 
able only  to  those  with  finite  alphabets.  In  [8]  ("Rate -Distort ion  functions 
of  Countably-Infinite  Alphabet  Memoryless  .Sources")  and  [27]  ("Moment  Space 
Upper  and  Lower  Error  Bounds  for  Digital  Systems  with  Intersymbol  Interfer- 
ence") , we  liave  shown  for  the  first  time  that  it  is  possible  to  use  the  Shan- 
non lower  bound  approach  for  the  countably-infinite  alphabet  sources.  We  used 
quite  advanced  functional  analysis  tools  of  the  contraction  mapping  theorem  and 
Schauder  fixed-point  theorem  for  this  problem.  In  [11]  ("Information  Rate  of 
a Memoryless  Gaussian  Source  under  an  Absolute  Magnitude  Criterion")  and  [21] 
(Evaluation  of  Rate-Distortion  Functions  for  a Class  of  I.T.D.  Sources  under 
an  Absolute  Magnitude  Distortion  Criterion") , we  have  shown  for  the  first 
time  that  it  possible  to  evaluate  the  rate-distortion  function  exactly 
for  this  class  of  continuous  sources,  including  the  Gaussian  source,  by  vari- 
ational argument  when  the  Shannon  lower  bound  is  never  equal  to  the  rate- 
distortion  function.  In  [17] ("Properties  of  Rate -Distort ion  Functions,  Part 
2:  Rate-Distortion  Functions  with  Absolute  Value  Distortion  Measure")  and  [38] 
("Evaluation  of  Rate-Distortion  Functions  for  I.l.D.  Sources  with  Support  on 
Subsets  of  IR  under  an  Absolute -Magnitude  Criterion"),  by  using  this  technique 
we  were  able  to  obtain  exactly  the  rate-distortion  function  of  a uniform 
source  and  a Poisson  distributed  source.  In  [13] ("Some  Conments  on  the  General- 
ized Shannon  Lower  Bound  for  Stationary  Finite -Alphabet  Sources  with  Memory") , 
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we  have  shown  that  the  most  general  published  results  on  this  topic  contained 
a basic  error.  We  have  then  shown  for  the  first  time  that  it  is  possible  to 
establish  conditions  for  the  equality  of  the  Shannon  lower  bound  for  a class 
of  stationary  infinite  memory  finite -alphabet  sources. 

Tn  [20]  ("A  Distortion-Measure  Sensitivity  .Analysis  of  the  Rate-Distor- 
tion  Function"),  we  considered  a completely  new  type  of  problem  in  which  the 
source  statistics  are  assumed  to  be  known  but  the  appropriate  distortion 
measure  to  be  used  is  assumed  to  be  not  completely  specified.  Thus,  we  con- 
sidered the  sensitivity  of  the  rate -distortion  function  in  terms  of  the  dis- 
tortion measure.  This  problem  is  philosophically  similar  to  the  class  of 
problems  in  which  the  distortion  measure  is  fixed  but  source  statistics  are 
not  completely  specified.  It  is  clear  that,  in  order  to  use  rate-distortion 
| theory  in  any  practical  sense,  one  must  be  able  to  understand  both  classes  of 

problems  since  the  source  statistics  are  never  known  completely  in  advance 
nor  does  there  exist  only  one  proper  distortion  measure  of  interest. 

In  [25]  ("Encoding  of  a Continuous  Source  by  Twisted  Modulation"),  the 
rate-distortion  function  of  a uniformly  distributed  source  under  the  mean- 
square  error  criterion  was  evaluated  by  numerical  technique.  Then  the  maxi- 
mum output  versus  input  SNR  of  an  analog  conmunication  system  with  bandwidth 
expansion  for  bandlimited  i.i.d.  uniformly  distributed  input  source  was  ob- 
tained. Some  preliminary  method  of  attack  on  obtaining  an  upper  bound  on 
the  mean-square  error  of  the  minimum  probability  of  error  estimator  was  also 
discussed. 

In  the  third  field  of  research  interest,  we  considered  models  of  stochas- 
tic processes  for  signals  and  noises.  In  [12]  ("A  Representation  Theorem  and 
its  Applications  to  Spherically- Invariant  Random  Processes"),  we  obtained  a 
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basic  representation  theorem  for  this  class  of  processes.  The  n^-order 
characteristic  functions  and  probability  densities  of  these  processes  have 
been  given  in  an  explicit  manner.  This  result  is  important  to  probability 
theory  since  the  classes  of  n^-order  probability  densities  given  in  a 
constructive  manner  are  quite  limited.  Furthermore,  this  representation 
theorem  showed  a fundamental  relationship  between  s.i.r.p.  and  the  random 
mixture  of  Gaussian  processes.  Our  results  are  more  general  than,  and  in- 
clude, all  previously  known  results  on  this  topic  (e.g.,  Vershik  of  U.S.S.R. , 
Picinbono  of  France,  Blake  of  Canada,  Copper  of  U.S.A.,  etc.).  It  also 
turned  out  that  very  low  frequency  (VLF)  atmospheric  noise,  relevant  to  sub- 
marine conmunications , can  be  modeled  by  mixtures  of  Gaussian  noise. 

The  theory  of  wide-sense  stationary  stochastic  processes  indexed  on 
the  integers  and  the  real  line  has  been  well  developed.  Many  results  for 
wide-sense  stationary  processes  are  still  valid  when  the  parameter  set  is 
taken  from  an  abstract  locally-compact  abelian  group  (LCAG) . On  the  other 
hand,  sampling  expansions  used  in  information  and  coirmunication  theory 
modeling,  which  were  originally  derived  for  bandlimited  deterministic  signals, 
have  been  extended  in  our  studies  to  bandlimited  wide-sense  stationarly  sto- 
chastic processes  on  the  real  line.  In  [14]  ("Series  Expansions  for  Random 
Processes  on  a LCAG")  and  [18]  ("Series  Expansions  and  Applications  of 
Second  Order  Processes  on  a Locally  Compact  Abelian  Group"),  we  obtained 
sampling  expansions  for  bandlimited  signals  and  wide-sense  stationary  stochas- 
tic processes  and  Karhunen-Loeve  type  expansions  on  an  abstract  LCAG.  Fur- 
thermore, reproducing  kernel  Hilbert  space  techniques,  originally  proposed 
by  Loeve  (1955)  and  Parzen  (1960)  for  wide-sense  stationary  stochastic  pro- 
cesses on  the  real  line,  and  by  Yao  ("Applications  of  RKHS  --  Bandlimited 
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Signal  Modals,"  Information  and  Control,  1967)  for  deterministic  signals, 
have  also  been  used  in  an  abstract  LCAG  setting.  These  expansion  results 
on  LCAG  are  not  only  of  academic  interest,  but  shed  some  light  on  Fourier- 
Walsh  sampling  expansions  used  in  some  practical  data  reduction  schemes. 

In  conclusion,  we  have  used  various  advanced  mathematical  techniques 
in  probability,  statistics,  and  functional  analysis  to  study  various  theor- 
etically and  practically  oriented  problems  in  comnunication  and  information 
theories  . ftir  results  have  contributed  to  the  growth  of  theoretical  under- 
standing of  these  problems.  Our  results  have  also  contributed  significantly 
to  the  practical  evaluation  of  commication  system  performance.  In  this 
contract  period,  over  a dozen  people  including  graduate  students,  faculty 
associates,  and  the  principal  investigator  have  been  supported.  Of  our 
Ph.D.  graduates:  Dr.  J.  Slaton  is  a highly  regarded  torpedo  designer  at 
Naval  Undersea  Center  at  San  Diego;  Dr.  H.  Tan  is  an  Assistant  Professor 
at  Princeton  University;  Dr.  Shamash  and  Dr.  Tobin  are  with  Hughes  Aircraft 
Company  at  Los  Angeles;  Dr.  D.  Hopkins  is  at  IBM  at  Thousand  Oaks,  Califor- 
nia; Dr.  J.  Kawamura  is  at  TRW  Systems  at  Redondo  Beach,  California.  This 
research  work,  performed  during  their  thesis  study,  has  contributed  signifi- 
cantly to  their  professional  growth.  In  this  period,  the  principal  investi- 
gator has  also  done  consulting  work  at  Bunker  Ramo  on  a sea-return  cable 
high  data  communication  system  sponsored  by  the  Department  of  Navy  and  at 
Hughes  Aircraft  Conpany  on  radar  signal  analysis  for  the  Naval  F14  aircraft. 
The  research  activities  performed  under  this  contract  also  contributed  sig- 
nificantly to  the  successful  completion  of  these  tasks. 


